Prospective study validating a multidimensional treatment decision score predicting the 24-month outcome in untreated patients with clinically isolated syndrome and early relapsing-remitting multiple sclerosis, the ProVal-MS study.

INTRODUCTION: In Multiple Sclerosis (MS), patients´ characteristics and (bio)markers that reliably predict the individual disease prognosis at disease onset are lacking. Cohort studies allow a close follow-up of MS histories and a thorough phenotyping of patients. Therefore, a multicenter cohort study was initiated to implement a wide spectrum of data and (bio)markers in newly diagnosed patients. METHODS: ProVal-MS (Prospective study to validate a multidimensional decision score that predicts treatment outcome at 24 months in untreated patients with clinically isolated syndrome or early Relapsing-Remitting-MS) is a prospective cohort study in patients with clinically isolated syndrome (CIS) or Relapsing-Remitting (RR)-MS (McDonald 2017 criteria), diagnosed within the last two years, conducted at five academic centers in Southern Germany. The collection of clinical, laboratory, imaging, and paraclinical data as well as biosamples is harmonized across centers. The primary goal is to validate (discrimination and calibration) the previously published DIFUTURE MS-Treatment Decision score (MS-TDS). The score supports clinical decision-making regarding the options of early (within 6 months after study baseline) platform medication (Interferon beta, glatiramer acetate, dimethyl/diroximel fumarate, teriflunomide), or no immediate treatment (> 6 months after baseline) of patients with early RR-MS and CIS by predicting the probability of new or enlarging lesions in cerebral magnetic resonance images (MRIs) between 6 and 24 months. Further objectives are refining the MS-TDS score and providing data to identify new markers reflecting disease course and severity. The project also provides a technical evaluation of the ProVal-MS cohort within the IT-infrastructure of the DIFUTURE consortium (Data Integration for Future Medicine) and assesses the efficacy of the data sharing techniques developed. PERSPECTIVE: Clinical cohorts provide the infrastructure to discover and to validate relevant disease-specific findings. A successful validation of the MS-TDS will add a new clinical decision tool to the armamentarium of practicing MS neurologists from which newly diagnosed MS patients may take advantage. Trial registration ProVal-MS has been registered in the German Clinical Trials Register, `Deutsches Register Klinischer Studien` (DRKS)-ID: DRKS00014034, date of registration: 21 December 2018; https://drks.de/search/en/trial/DRKS00014034.

MISM: A Medical Image Segmentation Metric for Evaluation of Weak Labeled Data.

Performance measures are an important tool for assessing and comparing different medical image segmentation algorithms. Unfortunately, the current measures have their weaknesses when it comes to assessing certain edge cases. These limitations arise when images with a very small region of interest or without a region of interest at all are assessed. As a solution to these limitations, we propose a new medical image segmentation metric: MISm. This metric is a composition of the Dice similarity coefficient and the weighted specificity. MISm was investigated for definition gaps, an appropriate scoring gradient, and different weighting coefficients used to propose a constant value. Furthermore, an evaluation was performed by comparing the popular metrics in the medical image segmentation and MISm using images of magnet resonance tomography from several fictitious prediction scenarios. Our analysis shows that MISm can be applied in a general way and thus also covers the mentioned edge cases, which are not covered by other metrics, in a reasonable way. In order to allow easy access to MISm and therefore widespread application in the community, as well as reproducibility of experimental results, we included MISm in the publicly available evaluation framework MISeval.

Long-term intra- and inter-individual biological variation of serum lipid of HIV-infected and uninfected men participating in the Los Angeles Multi-Center AIDS Cohort Study (MACS).

BACKGROUND: To assess the long-term biological coefficient of variation within individuals (CVI) and between individuals (CVG), effect of aging and cholesterol lowering drugs on blood levels of lipids in HIV-1-infected and -uninfected men. METHODS: Bloods were analyzed every six months over 17 years for total cholesterol (TC), triglycerides (TGs), high-density lipoprotein cholesterol (HDL-C) and low-density lipoprotein cholesterol (LDL-C) in 140 HIV-uninfected (38-66 years old) and 90 HIV-treated infected (48-64 years old) white Caucasian men to examine CVI, CVG, and the effect of cholesterol lowering drugs (CLDs) on lipid levels, and estimated changes per year of biomarkers. RESULTS: With exception of HDL-C, the long term CVI compared with CVG were higher for serum levels of TC, TGs, and LDL-C in both HIV-1 infected and uninfected men not taking CLDs. Excluding results of TGs in HIV positive men, the CVI compared with CVG were lower for serum levels of TC, HDL-C, and LDL-C in both groups not taking CLDs. There were significant (p < 0.05) differences in the median serum values of lipid biomarkers among 77 HIV negative men taking and 63 not taking CLDs. Also, with exception of HDL, there were significant (p < 0.05) differences in the median values of TC, TGs and LDL-C among 28 HIV positive men taking or not taking CLDs. CONCLUSION: Long term CVI and CVG of biomarkers will be useful for monitoring antiviral therapy side effects on lipid profiles in HIV-infected men. CVI of HIV-infected men for TC, TGs, HDL, LDL were higher significantly than CVI of HIV-uninfected men. Interestingly the long term CVI were higher than CVG for the men, who were on CLDs compared to men not on CLDs. The long-term pattern of CVI and CVG of lipid markers in both HIV-infected and uninfected men on CLDs differed from their short-term pattern.

Towards a guideline for evaluation metrics in medical image segmentation.

In the last decade, research on artificial intelligence has seen rapid growth with deep learning models, especially in the field of medical image segmentation. Various studies demonstrated that these models have powerful prediction capabilities and achieved similar results as clinicians. However, recent studies revealed that the evaluation in image segmentation studies lacks reliable model performance assessment and showed statistical bias by incorrect metric implementation or usage. Thus, this work provides an overview and interpretation guide on the following metrics for medical image segmentation evaluation in binary as well as multi-class problems: Dice similarity coefficient, Jaccard, Sensitivity, Specificity, Rand index, ROC curves, Cohen's Kappa, and Hausdorff distance. Furthermore, common issues like class imbalance and statistical as well as interpretation biases in evaluation are discussed. As a summary, we propose a guideline for standardized medical image segmentation evaluation to improve evaluation quality, reproducibility, and comparability in the research field.

MISeval: A Metric Library for Medical Image Segmentation Evaluation.

Correct performance assessment is crucial for evaluating modern artificial intelligence algorithms in medicine like deep-learning based medical image segmentation models. However, there is no universal metric library in Python for standardized and reproducible evaluation. Thus, we propose our open-source publicly available Python package MISeval: a metric library for Medical Image Segmentation Evaluation. The implemented metrics can be intuitively used and easily integrated into any performance assessment pipeline. The package utilizes modern DevOps strategies to ensure functionality and stability. MISeval is available from PyPI (miseval) and GitHub: https://github.com/frankkramer-lab/miseval.

DrNote: An open medical annotation service.

In the context of clinical trials and medical research medical text mining can provide broader insights for various research scenarios by tapping additional text data sources and extracting relevant information that is often exclusively present in unstructured fashion. Although various works for data like electronic health reports are available for English texts, only limited work on tools for non-English text resources has been published that offers immediate practicality in terms of flexibility and initial setup. We introduce DrNote, an open source text annotation service for medical text processing. Our work provides an entire annotation pipeline with its focus on a fast yet effective and easy to use software implementation. Further, the software allows its users to define a custom annotation scope by filtering only for relevant entities that should be included in its knowledge base. The approach is based on OpenTapioca and combines the publicly available datasets from WikiData and Wikipedia, and thus, performs entity linking tasks. In contrast to other related work our service can easily be built upon any language-specific Wikipedia dataset in order to be trained on a specific target language. We provide a public demo instance of our DrNote annotation service at https://drnote.misit-augsburg.de/.

Multi-Disease Detection in Retinal Imaging Based on Ensembling Heterogeneous Deep Learning Models.

Preventable or undiagnosed visual impairment and blindness affect billion of people worldwide. Automated multi-disease detection models offer great potential to address this problem via clinical decision support in diagnosis. In this work, we proposed an innovative multi-disease detection pipeline for retinal imaging which utilizes ensemble learning to combine the predictive capabilities of several heterogeneous deep convolutional neural network models. Our pipeline includes state-of-the-art strategies like transfer learning, class weighting, real-time image augmentation and Focal loss utilization. Furthermore, we integrated ensemble learning techniques like heterogeneous deep learning models, bagging via 5-fold cross-validation and stacked logistic regression models. Through internal and external evaluation, we were able to validate and demonstrate high accuracy and reliability of our pipeline, as well as the comparability with other state-of-the-art pipelines for retinal disease prediction.

Robust chest CT image segmentation of COVID-19 lung infection based on limited data.

BACKGROUND: The coronavirus disease 2019 (COVID-19) affects billions of lives around the world and has a significant impact on public healthcare. For quantitative assessment and disease monitoring medical imaging like computed tomography offers great potential as alternative to RT-PCR methods. For this reason, automated image segmentation is highly desired as clinical decision support. However, publicly available COVID-19 imaging data is limited which leads to overfitting of traditional approaches. METHODS: To address this problem, we propose an innovative automated segmentation pipeline for COVID-19 infected regions, which is able to handle small datasets by utilization as variant databases. Our method focuses on on-the-fly generation of unique and random image patches for training by performing several preprocessing methods and exploiting extensive data augmentation. For further reduction of the overfitting risk, we implemented a standard 3D U-Net architecture instead of new or computational complex neural network architectures. RESULTS: Through a k-fold cross-validation on 20 CT scans as training and validation of COVID-19, we were able to develop a highly accurate as well as robust segmentation model for lungs and COVID-19 infected regions without overfitting on limited data. We performed an in-detail analysis and discussion on the robustness of our pipeline through a sensitivity analysis based on the cross-validation and impact on model generalizability of applied preprocessing techniques. Our method achieved Dice similarity coefficients for COVID-19 infection between predicted and annotated segmentation from radiologists of 0.804 on validation and 0.661 on a separate testing set consisting of 100 patients. CONCLUSIONS: We demonstrated that the proposed method outperforms related approaches, advances the state-of-the-art for COVID-19 segmentation and improves robust medical image analysis based on limited data.

COVID-19 Image Segmentation Based on Deep Learning and Ensemble Learning.

Medical imaging offers great potential for COVID-19 diagnosis and monitoring. Our work introduces an automated pipeline to segment areas of COVID-19 infection in CT scans using deep convolutional neural networks. Furthermore, we evaluate the performance impact of ensemble learning techniques (Bagging and Augmenting). Our models showed highly accurate segmentation results, in which Bagging achieved the highest dice similarity coefficient.

Pruritus Intensity Scales across Europe: a prospective validation study.

BACKGROUND: Chronic pruritus (CP) is a subjective symptom, and it is necessary to assess its intensity with validated patient-reported outcome tools in order to allow determination of the treatment course. OBJECTIVES: So far, the itch intensity scales were validated in small cohorts and in single languages. Here, we report the validation of the numerical rating scale, the verbal rating scale and the visual analogue scale for the worst and average pruritus intensity in the last 24h in several languages across Europe and across different pruritic dermatoses. METHODS: After professional translation, the intensity scales were digitized for use as a tablet computer application. Validation was performed in clinics for Dermatology in Austria, France, Germany, Italy, Poland, Russia, Spain, Switzerland and Turkey. RESULTS: A total of 547 patients with contact dermatitis, chronic nodular prurigo, psoriasis vulgaris, lichen planus or cutaneous T-cell lymphoma were included. The intensity scales showed a high level of reproducibility and inter-correlations with each other. The correlation with the Dermatology Life Quality Index was weak to strong in nearly all countries and dermatoses with the exception of France and patients with chronic nodular prurigo, for which no statistically significant correlations were found. CONCLUSIONS: The numerical rating scale, the verbal rating scale und the visual analogue scales are valid instruments with good reproducibility and internal consistency in German (Germany, Austria, Switzerland), French, Italian, Polish, Russian, Spanish and Turkish for different pruritic dermatoses. VAS worst was the best reproducible and consistent measuring instrument in all countries.

Electronic Image Documentation of Patient Reported Outcomes Using Mobile Technologies.

Patient Reported Outcomes (PROs) provide essential clinical data for the diagnosis and treatment of patients. Mobile technologies enable rapid and structured collection of PROs with a high usability. MoPat is an electronic PRO system developed at the Münster University that enables patients to complete PROs in multiple languages. This research reports the further development of MoPat and the inclusion of features to document images electronically that will be evaluated in a multi-site clinical research.

Reviving 30 Year Old Technology: Lessons Learned from Transferring Patient Data Using Data Matrix Codes.

In patient care and medical research patient data often has to be transferred between different electronic systems. These systems can be very heterogeneous, sometimes even legacy systems, and thus, often do not support standardized interfaces for data transfer. Since nowadays barcode scanners are commonly used in clinical routine and smartphones are accessible to most patients, we implemented different interfaces based on Data Matrix codes to transfer patient data between several medical applications. Objective of this work is to show different use cases in which Data Matrix codes have been successfully applied and discuss the lessons we have learned during the process of implementation and practical usage.

Interoperability Improvement of Mobile Patient Survey (MoPat) Implementing Fast Health Interoperability Resources (FHIR).

Despite the advances in health information technology and the increasing usage of electronic systems, syntactic and semantic interoperability between different health information systems remains challenging. An emerging standard to tackle interoperability issues is HL7 FHIR, which uses modern web technologies for communication like Representational State Transfer. The electronic patient reported outcome system Mobile Patient Survey (MoPat) was adapted to support metadata import and clinical data export using HL7 FHIR. Thereby, the data models of HL7 FHIR and MoPat were compared and the existing import and export functions of MoPat were extended to support HL7 FHIR. A test protocol including eight test datasets to proof functioning of the new features was successfully conducted. In the near future, a real time searching toolbar of FHIR metadata resources will be integrated within MoPat. MoPat FHIR import and export functions are ready to be used in a clinical setting in combination with a FHIR compliant clinical data server.

A Smart Device System to Identify New Phenotypical Characteristics in Movement Disorders.

Parkinson's disease and Essential Tremor are two of the most common movement disorders and are still associated with high rates of misdiagnosis. Collected data by technology-based objective measures (TOMs) has the potential to provide new promising and highly accurate movement data for a better understanding of phenotypical characteristics and diagnostic support. A technology-based system called Smart Device System (SDS) is going to be implemented for multi-modal high-resolution acceleration measurement of patients with PD or ET within a clinical setting. The 2-year prospective observational study is conducted to identify new phenotypical biomarkers and train an Artificial Intelligence System. The SDS is going to be integrated and tested within a 20-min assessment including smartphone-based questionnaires, two smartwatches at both wrists and tablet-based Archimedean spirals drawing for deeper tremor-analyses. The electronic questionnaires will cover data on medication, family history and non-motor symptoms. In this paper, we describe the steps for this novel technology-utilizing examination, the principal steps for data analyses and the targeted performances of the system. Future work considers integration with Deep Brain Stimulation, dissemination into further sites and patient's home setting as well as integration with further data sources as neuroimaging and biobanks. Study Registration ID on ClinicalTrials.gov: NCT03638479.

Cross-European validation of the ItchyQoL in pruritic dermatoses.

BACKGROUND: Chronic pruritus (CP) is a frequently occurring symptom in inflammatory dermatoses, causing a high burden and limitations to health-related quality of life (HRQoL). OBJECTIVE: The ItchyQoL was developed to assess the impairment to HRQoL in patients with CP. However, it has only been validated in English and German. Here, we report the validation in several languages across Europe. METHODS: After professional translation, the versions of ItchyQoL were digitized for use as a tablet application. Validation was performed in clinics for dermatology in Austria, France, Germany, Italy, Poland, Russia, Spain, Switzerland and Turkey. RESULTS: Five hundred and thirty-five patients with either contact dermatitis, chronic prurigo - nodular type, psoriasis vulgaris, lichen planus or mycosis fungoides/Sézary syndrome and with CP ≥ 3 on the numerical rating scale were included. ItchyQoL showed a high level of consistency (Cronbach's-α, all: 0.95) and test-retest reliability (intraclass correlation: 0.91). It strongly correlated with the Dermatology Life Quality Index (r = 0.72, P < 0.001) and moderately correlated with itch intensity scales in the study population (visual analogue scale r = 0.46; numerical rating scale r = 0.51; verbal rating scale r = 0.51, for all: P < 0.001). CONCLUSION: ItchyQoL is now also validated in French, Italian, Polish, Russian, Spanish and Turkish and can be used in clinical trials in countries speaking these languages.

Electronic Collection of Multilingual Patient-Reported Outcomes across Europe.

BACKGROUND: Patient-reported outcomes (PROs) are information provided directly by patients that helps in improving patient diagnosis and treatment. Validated translations of PROs can be used to treat international patients. Electronic systems and especially mobile devices provide a great opportunity for their collection; however, these systems are normally study-oriented and therefore single language, not scalable, and not interoperable. OBJECTIVES: This article reports the development of a multicenter, multilingual, and interoperable electronic PRO (ePRO) system and evaluates its user satisfaction in an international clinical study. METHODS: The ePRO named "MoPat2" was developed using Java 8 and jQuery Mobile 1.4.5. The system was evaluated in the context of the European dermatology project "European Network on Assessment of Severity and Burden of Pruritus"(PruNet), which aimed to unify the assessment of itch in routine dermatological care in Europe. Twenty-six clinicians and 468 patients from 8 European clinical centers were asked to complete a user satisfaction questionnaire regarding the use of MoPat2 with a tablet personal computer. The results were then analyzed and correlated with the age, gender, and language of the respondents. RESULTS: MoPat2 was enhanced with multilingual capabilities and is now able to conduct surveys in several languages, as well as store and display the results in the local language. The interviewed clinicians rated the system with an average score of 2.0 ("good") in a 1 to 5 Likert scale. Note that 93.9% of the patients (439 of 468) reported having got on well using the system and 88.9% (416 of 456) would be willing to further use it. The age of the patients not willing to further use MoPat2 was, in average, considerably higher than the age of patients willing to use the system. CONCLUSIONS: This study represents the first use of an ePRO system for the collection of multilingual PROs in an international, multicenter setting. MoPat2 has been evaluated by both clinicians and patients in the context of a European dermatological study, resulting in a high user satisfaction. The system will be further developed to include new features such as patient follow-ups outside of the clinical setting.

Conducting a Multilingual Study Researching Traumatised Refugees Utilizing a Patient-Reported Outcome System.

Approximately 300,000 asylum-seeking children arrived in Europe in 2015. The chance of experiencing a traumatic event is very high for fleeing children. Since the origin of the refugees is widespread, the languages spoken are diverse. Multilingual electronic patient-reported outcome systems (ePROs) can be used to gather medical data in a foreign language and display the results in the health professionals' language, which helps overcoming the language barrier. Utilizing such a system, a two-phase study aiming to screen refugee minors for potential mental health issues has started. Potential eligible participants are examined using questionnaires with good psychometric properties and cross-cultural applicability. To date, 75 minors and 21 of their relatives participated in the study, being German and Arabic the most desired languages for the electronic survey. Developing a system that provides multilingual questionnaires entails several drawbacks like a cumbersome translation process and dealing with writing directions. The proposed translation process and the ePRO can be re-used in similar studies.

Humanistic burden of chronic pruritus in patients with inflammatory dermatoses: Results of the European Academy of Dermatology and Venereology Network on Assessment of Severity and Burden of Pruritus (PruNet) cross-sectional trial.

BACKGROUND: Chronic pruritus is a multifactorial, challenging symptom of global relevance. OBJECTIVE: The European Academy of Dermatology and Venereology Network on Assessment of Severity and Burden of Pruritus (PruNet) investigation aimed to analyze the severity and humanistic burden of chronic pruritus in patients suffering from inflammatory dermatoses across Europe. METHODS: Prospectively collected routine data on 552 patients (with atopic dermatitis, contact dermatitis, prurigo nodularis, psoriasis vulgaris, lichen planus, or mycosis fungoides [pruritus numeric rating scale score ≥3]) from 9 European centers (in Austria, France, Germany, Italy, Poland, Russia, Spain, Switzerland, and Turkey) were analyzed by univariate and multivariate variance analyses of various itch characteristics and quality of life (as measured by the Dermatology Life Quality Index and the ItchyQoL). RESULTS: Duration, frequency, and intensity of pruritus (according to a numeric rating scale and visual analog scale) and related impairment of quality of life differed between European centers and dermatologic diagnoses (P < .05). The country in which the center was located had a greater impact on how patients evaluated pruritus intensity and quality of life than diagnosis did (P < .001). LIMITATIONS: One center per country was included. CONCLUSION: The humanistic burden of chronic pruritus in patients with inflammatory dermatoses is high. European cross-cultural factors may have a stronger influence than a specific dermatologic diagnosis on how patients rate intensity of pruritus and quality of life.

Standardising the Development of ODM Converters: The ODMToolBox.

CDISC's Operational Data Model (ODM) is a flexible standard for exchanging and archiving metadata and subject clinical data in clinical trials. The Portal of Medical Data Models (MDM-Portal) uses ODM to store more than 15000 medical forms. As not every electronic health system accepts ODM as input format, there is a need for conversion between ODM and other data standards and formats. This research proposes a standardised template-based process to develop ODM converters. So far, ten converters have been developed and integrated in the MDM-Portal following this process and new ones should be included soon. The template, programming utilities and an ODM test suite have been made online available and can be used to easily develop new converters.

Validation of Pruritus Measures Gathered with the Electronic Patient-reported Outcome System MoPat.

In order to improve diagnosis and treatment, physicians require information about the social context and quality of life of their patients. The Center for Chronic Pruritus at the University Hospital Münster achieves this goal using the electronic patient-reported outcome system "Mobile Patient Survey", which assesses pruritus and quality of life measures. The aim of this study is to evaluate the consistency and reliability of such measures. A total of 42 patients, age range 19-82 years, participated in the study and were asked to assess the measures at baseline via a paper questionnaire, and to use the "Mobile Patient Survey" at baseline and after 1 h in order to test reliability. Statistical analysis was performed using coefficient rc for metric variables and weighted kappa κw for categorical variables. The internal consistency of all measures was unaffected. It was shown that 6 out of 7 measures can be assessed without loss of reliability. It is recommended that questionnaires for electronic usage are assessed for validity and reliability.